Learning Objectives

After completing this lesson, you’ll be able to:

• Identify which aspects of output data to inspect when problems occur.
• Use record counts to locate problems in a workspace.
• Inspect rejected feature attributes to determine why a feature failed.

Instructions

In this lesson, you will:

• Optional: Watch the Use Output and Feature Counts to Identify Problems video.
  • This video uses an older title for this lesson, but the content is otherwise mostly unchanged.
• Scroll down to read the text below.
• Optional: Let us know if you found this lesson relevant to your role by filling out the survey at the bottom of the page.
• Click 'Next' to mark the lesson complete.

Resources

• Starting workspace 
  • C:\FMEData\Workspaces\UseDataIntegrationBestPractices\use-output-and-record-counts-to-identify-problems.fmw
• Complete workspace
  • C:\FMEData\Workspaces\UseDataIntegrationBestPractices\use-output-and-record-counts-to-identify-problems-complete.fmw
• VancouverNeighborhoods.zip (OGC / Google KML)
  • C:\FMEData\Data\Boundaries\VancouverNeighborhoods.kml
• street-trees.csv
  • C:\FMEData\Resources\FormBasic\street-trees.csv

Inspecting Output

Even if a workspace ran to completion without warnings or errors, it does not follow that the output matches expectations or requirements. For whatever reason, the workspace may be producing data incorrectly. We can determine this by inspecting the translation output.

To inspect your output, view it in Data Preview or the destination application.

You can inspect several aspects of data, including:

• Format: Is the data in the expected format?
• Schema: Is the data subdivided into the correct layers, categories, or classes?
• Geometry: Is the geometry in the correct spatial location? Are the geometry types suitable?
• Symbology: Is each feature's color, size, and style correct?
• Attributes: Are all the required attributes present? Are all integrity rules being followed?
• Quantity: Does the data contain the correct number of features?
• Output: Has the translation process restructured the data as expected?

When encountering problems, you should inspect your data to see if its components are incorrect.

This stage is solely to determine if there are any problems.

Record Counts

A workspace record count refers to the numbers shown on each connection once a translation is complete:

Once you find an error or problem, record counts help us identify where that problem occurred.

In the above screenshot, if the Clipper output is incorrect, you should inspect the prior record counts to see if any counts look wrong. Perhaps you know that there are seven neighborhoods, but the record count shows only six.

Incorrect Output

You can check several things when the number of output features is incorrect.

If you get zero output, and the record counts show that all features entered a transformer, but none emerged, then you can be reasonably confident that the transformer is the cause of the problem:

For example, 80 features enter the Clipper transformer (to be clipped against a single boundary), but none emerge. The Clipper transformer is almost certainly the cause of any incorrect output.

The transformer does not reject the data; it merely does not pass the test expected. It's possible that Clipper and Clippee don't occupy the same coordinate system; hence, one does not fall inside the other.

Turning on data caching helps to confirm this to be the case:

Alternatively – and this is a common cause of missing features – the author has connected the wrong output port! For example, this user connected the StatisticsCalculator Summary output port when they wanted the features from the Complete port:

You can observe record counts and data caches to ensure you are getting the expected number of features and they look correct.

Rejected Features

Sometimes, when features go missing, they are rejected by a transformer. Many transformers include a <Rejected> port to output these invalid features:

Remember, features are automatically counted and stored on a <Rejected> port, even if data caching is off.

As an additional benefit, the rejected features will often include a rejection code attribute explaining the problem:

Simply reading a rejection code can often help you identify the problem. Please refer to the transformer documentation for more information if it does not.

Checking Key Stages

 If the record counts cannot help pinpoint a problem's location, the next step is inspecting data at crucial translation stages.

Generally, issues in an output dataset occur when:

• The workspace reads data incorrectly.
• The workspace transforms data incorrectly.
• The workspace writes data incorrectly.
• Another application misinterprets the data.

Exercise

Amar is having problems with a workspace that clips street trees to Vancouver neighborhood boundaries and sends it to Frank for help. Frank uses his debugging skills to track down two separate issues: a rejected feature that stops the translation, and a coordinate system mismatch that produces empty output.

In this exercise, you will:

• Use the Translation Log to identify the cause of a rejected feature.
• Inspect data caches to confirm a coordinate system mismatch.
• Fix both issues to produce correct output.

1) Open Starting Workspace

• Start FME Workbench (2026.1 or later).
• Open your starting workspace (C:\FMEData\Workspaces\UseDataIntegrationBestPractices\use-output-and-record-counts-to-identify-problems.fmw).
• Click Run.
  • You'll notice that the workspace stops running before it's done.
• In the Unexpected Input window, click OK.
  
  • The Unexpected Input window appears when FME reads data not included in the workspace. This can happen with formats like KML, where the entire dataset is read even if only one feature type is on the canvas. You can safely ignore it once you confirm you have all the source data you need.

2) Find Rejected Feature Code

Use the Translation Log to find out why the workspace stops before completing due to an error.

• In the Translation Log, click Errors.

• Review the error message:
  • The below feature caused the translation to be terminated
  • JSONExtractor_<Rejected> (TeeFactory): JSONExtractor_<Rejected>: Termination Message: 'JSONExtractor output a <Rejected> feature.  To continue translation when features are rejected, change 'Workspace Parameters' > Log and Troubleshoot > 'Rejected Feature Handling' to 'Continue Translation''

  • The error message points to a rejected feature from the JSONExtractor. The error itself does not explain the cause. You need to inspect the rejected feature directly to find out more.
• On the canvas, locate the JSONExtractor.
• On the <Rejected> port, click the data cache icon.

• In Table View, scroll right to find the fme_rejection_code attribute.
  • The value is NO_RESULT.
• Check the Geom attribute.
  • The value is <null>. You'll need to find out why the JSONExtractor returned no result for this feature.

3) Find Cause of Rejected Feature

Now that you know which record was rejected, check the JSONExtractor transformer parameters to determine why.

• Double-click the JSONExtractor.
  • In the parameters, you can see the transformer reads JSON from the Geom attribute.
  • It also extracts latitude and longitude coordinates using a JSON Query.
  • If Geom is null, the transformer has nothing to read and rejects the record.

• Click Ok.
• In the JSONExtractor, click the Evaluated data cache.
• Inspect the Geom, lat, and long attributes.
  • The three records that processed successfully all have a value for Geom, and therefore values for lat and long. This confirms the rejected record failed because Geom was null.

4) Fix Rejected Feature

To fix this problem, filter out null Geom records before they reach the JSONExtractor.

• Add a Tester after the CSV reader feature type.
• Configure the Tester with the following test clause:
  • Left Value: Geom
  • Operator: Attribute Has a Value

• Connect the Passed port to the JSONExtractor input.
• Run the workspace.
  • The JSONExtractor should no longer output rejected records.

5) Use Record Counts to Debug

The workspace runs without errors, but you notice something looks off. Check the Clipper transformer's record counts to see if the results look as expected.

• On the Clipper's output ports, inspect the record counts.

• You notice something unexpected:
  • The Inside port is empty and all records are coming out of the Outside port.
  • This means the Clipper did not detect any overlap between the tree points and the neighborhood polygons.
  • When all records exit the Outside port, the most common cause is a coordinate system mismatch. Check both datasets before drawing any conclusions.

6) Investigate Coordinate Systems

You’ve got the skills to troubleshoot this one. Before following the steps below, take a minute to see if you can confirm whether a coordinate system mismatch is causing the overlap issue.

Open both datasets in Data Preview at the same time. This makes them visible together in the Graphics View, so you can check whether they overlap.

• On the Clipper, click the Outside port data cache.
• Ctrl+click the Neighborhoods' port data cache.
• Select a tree data record.
• In the Record Information Window, check the Coordinate System.
  • The tree data is in UTM83-10.

• Switch to the Neighborhoods table and select a tree data record.
  • The Neighborhoods data is in LL84. This confirms a coordinate system mismatch.

7) Fix Coordinate Systems

Both datasets need to be in the same coordinate system before the Clipper can detect overlap. It is better to conduct spatial analysis using a projected coordinate system, so reproject the Neighborhoods data to UTM83-10 to match the tree data. You can always reproject back to LL84 at the end if needed.

• Right-click the Reprojector and select Duplicate.
• Drag the duplicated Reprojector onto the Neighborhoods stream until it connects to the line, then release.

• Run the workspace.
• Inspect the record counts on the Clipper's output ports.
  • The Inside port should now contain records, confirming the coordinate system mismatch has been resolved.

Optional Challenge: Why Use a Projected Coordinate System?

You fixed the coordinate system mismatch by duplicating the Reprojector to reproject the Neighborhoods data to UTM83-10. But there was another option: removing the Reprojector entirely so both datasets stay in LL84. Both approaches resolve the mismatch, but one is the better choice for spatial analysis. Consider why before looking at the answer.

In this challenge, you will:

• Explain why reprojecting both datasets to UTM83-10 is a better choice than leaving both in LL84.

Leave Us Feedback on This Lesson